The same way as in creating new identity, you have to select the desired identity first and enter the correct password to the text input field.
If you now click the button for identity removal, the identity will be removed.